Concerning the problems that the traditional Convolutional Neural Network (CNN) ignores the contextual semantics of words when performing sentiment analysis tasks and CNN loses a lot of feature information during max pooling operation at the pooling layer, which limit the text classification performance of model, a parallel hybrid neural network model, namely CA-BGA (Convolutional Neural Network Attention and Bidirectional Gated Recurrent Unit Attention), was proposed. Firstly, a feature fusion method was adopted to integrate Bidirectional Gated Recurrent Unit (BiGRU) into the output of CNN, thus semantic learning was enhanced by integrating the global semantic features of sentences. Then, the attention mechanism was introduced between the convolutional layer and the pooling layer of CNN and at the output of BiGRU to reduce noise interference while retaining more feature information. Finally, a parallel hybrid neural network model was constructed based on the above two improvement strategies. Experimental results show that the proposed hybrid neural network model has the characteristic of fast convergence, and effectively improves the F1 value of text classification. The proposed model has excellent performance in Chinese short text sentiment analysis tasks.